No More Pesky Learning Rates

The performance of stochastic gradient descent (SGD) depends critically on how learning rates are tuned and decreased over time. We propose a method to automatically adjust multiple learning rates so as to minimize the expected error at any one time. The method relies on local gradient variations across samples. In our approach, learning rates can increase as well as decrease, making it suitable for non-stationary problems. Using a number of convex and non-convex learning tasks, we show that the resulting algorithm matches the performance of SGD or other adaptive approaches with their best settings obtained through systematic search, and effectively removes the need for learning rate tuning

Phase Harmonic Correlations and Convolutional Neural Networks

A major issue in harmonic analysis is to capture the phase dependence of frequency representations, which carries important signal properties. It seems that convolutional neural networks have found a way. Over time-series and images, convolutional networks often learn a first layer of filters which are well localized in the frequency domain, with different phases. We show that a rectifier then acts as a filter on the phase of the resulting coefficients. It computes signal descriptors which are local in space, frequency and phase. The non-linear phase filter becomes a multiplicative operator over phase harmonics computed with a Fourier transform along the phase. We prove that it defines a bi-Lipschitz and invertible representation. The correlations of phase harmonics coefficients characterise coherent structures from their phase dependence across frequencies. For wavelet filters, we show numerically that signals having sparse wavelet coefficients can be recovered from few phase harmonic correlations, which provide a compressive representationComment: 26 pages, 8 figure

Statistical learning of geometric characteristics of wireless networks

International audienceMotivated by the prediction of cell loads in cellular networks, we formulate the following new, fundamental problem of statistical learning of geometric marks of point processes: An unknown marking function, depending on the geometry of point patterns, produces characteristics (marks) of the points. One aims at learning this function from the examples of marked point patterns in order to predict the marks of new point patterns. To approximate (interpolate) the marking function, in our baseline approach, we build a statistical regression model of the marks with respect some local point distance representation. In a more advanced approach, we use a global data representation via the scattering moments of random measures, which build informative and stable to deformations data representation, already proven useful in image analysis and related application domains. In this case, the regression of the scattering moments of the marked point patterns with respect to the non-marked ones is combined with the numerical solution of the inverse problem, where the marks are recovered from the estimated scattering moments. Considering some simple, generic marks, often appearing in the modeling of wireless networks, such as the shot-noise values, nearest neighbour distance, and some characteristics of the Voronoi cells, we show that the scattering moments can capture similar geometry information as the baseline approach, and can reach even better performance, especially for non-local marking functions. Our results motivate further development of statistical learning tools for stochastic geometry and analysis of wireless networks, in particular to predict cell loads in cellular networks from the locations of base stations and traffic demand

Kymatio: Scattering Transforms in Python

The wavelet scattering transform is an invariant signal representation suitable for many signal processing and machine learning applications. We present the Kymatio software package, an easy-to-use, high-performance Python implementation of the scattering transform in 1D, 2D, and 3D that is compatible with modern deep learning frameworks. All transforms may be executed on a GPU (in addition to CPU), offering a considerable speed up over CPU implementations. The package also has a small memory footprint, resulting inefficient memory usage. The source code, documentation, and examples are available undera BSD license at https://www.kymat.io

Complete Genome Sequencing of Lactobacillus plantarum ZLP001, a Potential Probiotic That Enhances Intestinal Epithelial Barrier Function and Defense Against Pathogens in Pigs

The full text of this article can be freely accessed on the publisher's website

Swine-Derived Probiotic Lactobacillus plantarum Inhibits Growth and Adhesion of Enterotoxigenic Escherichia coli and Mediates Host Defense

Weaning stress renders piglets susceptible to pathogen infection, which leads to post-weaning diarrhea, a severe condition characterized by heavy diarrhea and mortality in piglets. Enterotoxigenic Escherichia coli (ETEC) is one of typical strains associated with post-weaning diarrhea. Thus, prevention and inhibition of ETEC infection are of great concern. Probiotics possess anti-pathogenic activity and can counteract ETEC infection; however, their underlying mechanisms and modes of action have not yet been clarified. In the present study, the direct and indirect protective effects of Lactobacillus plantarum ZLP001 against ETEC infection were investigated by different methods. We found that bacterial culture and culture supernatant of L. plantarum ZLP001 prevented ETEC growth by the Oxford cup method, and ETEC growth inhibition was observed in a co-culture assay as well. This effect was suggested to be caused mainly by antimicrobial metabolites produced by L. plantarum ZLP001. In addition, adhesion capacity of L. plantarum ZLP001 to IPEC-J2 cells were observed using microscopy and counting. L. plantarum ZLP001 also exhibited a concentration-dependent ability to inhibit ETEC adhesion to IPEC-J2 cells, which mainly occurred via exclusion and competition mode. Furthermore, quantitative real time polymerase chain reaction (qPCR) analysis showed that L. plantarum ZLP001 upregulated the expression of host defense peptides (HDPs) but did not trigger an inflammatory response. In addition, L. plantarum ZLP001 induced HDP secretion, which enhanced the potential antimicrobial activity of IPEC-J2 cell-culture supernatant after incubation with L. plantarum ZLP001. Our findings demonstrate that L. plantarum ZLP001, an intestinal Lactobacillus species associated with piglet health, possesses anti-ETEC activity. L. plantarum ZLP001 might prevent ETEC growth, inhibit ETEC adhesion to the intestinal mucosa, and activate the innate immune response to secret antimicrobial peptides. L. plantarum ZLP001 is worth investigation as a potential probiotics